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(54) Apparatus and method for demultipiexing multiplexed data 



(57) Disclosed is an apparatus t or demultiplaxinQ 
multiplexed data . This apparatus comprises a demulti- 
plexer for receiving a multiplexed packet stream' In 
which a plurality of ot^'ect dat a are multiplexed, and 
d emultiplexino the multiplexed data with reference to 
packet i dentification numbers added to the re^ective 
packets; a decoder Ibr de oodino M piec es of object data 
se parated from the multiplexed d ata, and outputting M 
pieces of decoded object data; a memory for storing N 
pieces of information relating to a program and sepa- 
rated by the demultiplexer; a memory for Coring object 
compositio n i nformatio n separated by the demulti- 
plexer; a memory for storing i nfornrmtion relating to th e 
object data and separated by the demultiplexer; a com- 
positor for compositing the M pieces of decoded object 
data; and an information a nalyzer far analyzin g the 
p acket Identrf icatio n numbers from the N pieces of infor - 
mation relating to the program, the object composition 
information, or the information relating to the object 
data. Therefore, when a multiplexed packet stream in 
which MPEQ4 data are multiplexed by MPEQ2 is 
d emultiplexed , the MPEG4 data can be s eparated and 
de coded with high reliablity. 
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Description 

FIELD OF THE INVENTION 

The present invention relates to an apparatus and a s 
method for d emultiplexing a oadtet stream in which var^ 
ious kinds of data are multiplexed, and a data stora ge 
mediu m containing a p rogram for impiementinQ the 
de multiplexi ng process by software . AAore specifically,-^ 
the invention relates to demultiplexing of a packet 10 
stream in which various kinds of data are multiplexed by 
MPEG2 which Is an international standard of Image 
compressive coding, or a p acket stream In which a p lu- 
r ality of object dat a are muttiplexed by MPEQ4 which Is 
currently being standarcfized. is 

BACKGROUND OF THE INVENTION 

In recent years. MPEG4 has been discussed as a 
method for transmitting digital data of videa audio, and 20 
control information in packets (reference literatures: 
ISO/IEC JTC1/SC29/WG11 N1692 MPEG4-Syslems 
Working Draft 4.0, ISO/IEC JTC1/SC29/WG11 N1693 
MPEG4-Systems Verification Model 4.0). In MPEG4. 
digital data of oompressively coded video and audio, 2s 
object composition information descrit»ng hierarchy of 
objects, and object descrqptors describing coding 
schemes of the objects are subjected to the MPEG4 
packetization using the Access Unit Layer, and these 
MPEG4 packets are further packetized by another mul- 30 
tiplexing protocol such as MPEG2 (reference literature: 
ISO/IEC JTCiySC29/WGl1 N801 "ISO/IEC 13818-1). 
On the other hand, in MPEQ2. video and audio data _ 
oompressively coded by MPEG2 are packetized in a for- 
mat called "PES (Packetized Elementary Stream) 
Packet", and the PES packet is divided into fixed-length 
packets called TS (T ransport Stream) packets ". 

In MPEG4. standardization for which ts cunrentiy in 
progress, since the MPEG4 packetization using the 
Access Unit Layer is indispensable, video and audio 
data compressively coded by MPEG2 are also sub- 
jected to the MPEG4 padcetization. Therefore, when an 
MPEG2 data receiver for receiving satellite broadcast- 
ing or CATV broadcasting, which have been spread in 
recent years, r epeivp^ pn MPEG4 data stream in whic h 46 
v ideo and audio data compressively ooded by MPEG2 
ar e MPEG4-packetized and multiplexed, the receiver 
cannot separate and decode the video and audio data 
compressed by MPEG2. 

Furthermore, a broadcasting receiving terminal so 
receives object composition information desaibing hier- 
archy of objects, arrd object descriptors describing cod- 
ing schemes of the objects, several times, even though 
the conterrts of these data are not changed. TNs repeti- 
tion is necessary because tiiere is a possibility that ss 
receiving is started at an aibitrary timing in the broad- 
casting receiving terminal. The object composition infor- 
mation and tiie object desaiptors are also subjected to 
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MPEG4 pad(etization using the Access Unit Layer as 
well as video and audio data. However, when the receiv- 
ing termlr^l receives such data of the same contents 
repeatedly, the burden on the receiving terminal may be 
increased. 

Furthermore. MPEG4 does not provide a method 
for specifying object composition information, object 
descriptors, and video and audio data conrpressively 
ooded by MPEG4, from multiplexed data. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide an 
apparatus and a method for d emultiplexing multiplexe d 
data, that enable reliable separation and d ecodin g of 
MPEG4 data even through the MPEG4 data is multi- 
plexBdbyMPEQ2. 

Other objects and advantages of the invention will 
become apparent from the detailed description that fol- 
lows. The detailed description and specific embodi- 
ments described are provided only for illustration since 
various additions and modifications within the spirit and 
scope of the invention will be apparent to tiiose skilled in 
the art from the detailed description. 

According to a first aspect of the present invention, 
there Is provided a demuftiplexinq apparatu s compris- 
ing: d enruiltiplexinq mear^ p for r eceiving a multiplexe d 
packet stream I n which a plurality of object data are mut- 
tiplexed. and d emultiplexing the multiplexed ^ ata with 
reference to p acket identificatio n numbers added to tiie 
respective packets; d ecoding nrteans for decoding M 
piec es of objecL dgta sep arated from the multiplexe d 
-data, and outputting M pieces of decoded object a 
memory for storing N pieces of information relating to a 
program smd separated by the demultiplexing means: a 
menmry for storing ok^ect composition infbrn^tion sep- 
arated by the demuftiptexing means; a memory for stor- 
ing information relating to the object data and separated 
t)y the demultiplexing means; composition means for 
compositing the M pieces of decoded object data; and 
in formation analys is means for a nalyzing the pack et 
id entification numbers from the N pieces of information 
relating to tiie program, th e object composition infom ia- 
tion. or the information relatinp to the obiect dat a. 

According to a second aspect of the present inven- 
tion, in tiie above-described d emultiplex; ing apparatigs, 
the information analysis means analyzes the N pieces 
of information relating to the program, obtains a packe t 
ide ntification numt wr PID_syn of a packet containing 
the object c omposition information , and outputs the 
padtet identification number PID_syn to the demulti- 
plexing means; and the demultiplexin g means sepa - 
rates the p acket containing the object compositio n 
in formatio n fr om the input multiplexed packet strea m 
with reference to the packet i dentificatign nunrrb er 
PIP^^^. arxt outputs this packet. 

According to a third aspect of the present invention, 
in the at>ove-described demultiplexing apparatus, the 
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information analysis means analyzes tlie N pieces of 
information relating to the program, o btains a packe t 
id entification numt)er PID„od of a pack et containing the 
information relating to the oDject data, and outputs the 
packet Identification n umber PIDjod to the demultiplex- 
ing means; and the demultiplexing means_separates the 
pack et containing the infonmation relating to the objec t 
data fro m the input multiplexed pack et stream with refer- 
ence to the packet identification n umber PID_od, and 
outputs this packet 

According to a fourth aspect of the present inven- 
tion, in the above<iescribed demultiplexing apparatus, 
the infornrHtion analyst means analyzes the object 
composition information, and outputs Information indi- 
cating tiie hierarchical relationship of the object data 
toward the composition means; and the composition 
means composites the 1^ pieces of decoded object data 
with reference to the hierarchical relationship. 

According to a f ifth aspect of the present invention, 
in tiie above-desaibed demultiplexing apparatus, tiie 
information analyse means analyzes the information 
relating to tiie object data, judges a o ode Al_predefine d 
for each of tiie object data , which oode indicate s 
whether packetization according to the internatio nal 
st andard MPEG4 exists or not, and outputs the code 
AljMredefined toward tiie demultiplexing means; and 
the demultiplexing means p erforms separation of th e 
MPE_G4 packetization for each object data with refe r- 
ence to the c ode Aljxeclefined. 

According to a six^t ] aspec t of the present invention, 
in tiie above^Jesaibed demultiplexing apparatus, tiie 
information analysis means a nalyzes the informatio n 
r elating to ttieobiectjdat a. ju dges a oode Al predefine d 
fo r each of tiie object da ta, whrch code indicate s 
wh ether packetization ■^ccQcdiD aJta--iheJoternati.oaal 
standard^ MPEG4 exist s or not, and outputs the code 
Aljuredefined toward the decodir^ means; and tiie 
decoding means o erforms separation of the MPEG 4 
p acketization with reference to the code Al jxedef ine d. 

According to a s eventii asp ect of the present Inven- 
tion, in the above-desaibed demultiplexing apparatus, 
tiie information analysis means analyzes the informa- 
tion relating to the object data, judges a code Al_conf Ig 
fo r each of the obiect data , which c ode indicates^ ettiOQ 
info rmation of a packet header accordino to the intern a- 
tio nal standard MPEG4 , and outputs the c ode AI_oonf ig 
toward the denultiplexing means; and the d emultiple x- 
ing means performs separation of the MPEG4 packe ti- 
z ation for each_ob lecLdata witii reference to tiie code 
Al^ config . 

According to an eiphth aspect c* the present inven- 
tion, in the above-desaibed denrujitiplexing apparatus, 
the informatiori analysis means analyzes the informa- 
tion relating to tiie object data, judges a code ALconf ig 
for each of the object data, which code indicates setting 
information of a packet header according to the interna- 
tional standard MPEQ4, and outputs the code AI_conf ig 
toward the decoding means; and the decoding means 
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performs separation of ttie MPEG4 packetization witii 
reference to the code AI_config. 

According to a riinth aspect of the present inven- 
tion, in the above-descrbed demultiplexing apparatus, 
tiie information analysis means analyzes the N pieces 
of information relating to tiie program, o btains a pack et 
ide ntification number PID_d of a packet containina. ttie 
object da ta and a pa dcet identiflcatlonjiumber FMCLjd 
according to tiie international s tandard f^PEG 4. and 
outputs these Id entification numb ers toward tiie demul- 
tiplexing means; and tiie de multiplexing means se pa- 
rates the pad<et containing the obiect data from_fli e 
input nultiplexed packet stream wi th reference to the 
ide ntification nunrtoers PIP d and FI^C d. and outputs 
thepadgt 

According to a t enth aspe ct of the present Inven- 
tion, in the above-descrbed demultiplexing apparatus, 
tiie Information analysis means obtains a oode for 
deciding whether tiie pa cket identifteation numb er 
F MCjd according to tiie intern ational standard_MPEG4 
is added or not. by analyzing the N pieces of Information 
relating to tiie program. 

According to an eleventh aspect of tiie present 
invention, in tiie above-described demultiplexing appa- 
ratus, the information analysis means receives tiie 
otjject composition information which is divided Into at 
least one section, and integrates these sections with 
reference to v erston num bers and section numbers 
w hich are added at ttie heads of tiie respective sec- 
tions, tiiereby obtaining the object composition Informa- 
tion. 

According to a twelfth aspect of the present inven- 
tion, In the atjove-descrbed demultiplexing apparatus, 
the infbrrrration analysis means receives the information 
relating to the object data, which information Is divided 
Into at least one section, and Integrates these sections 
with reference to version numbers and section numbers 
which are added at tiie heads of the respective sec- 
tions, thereby obtaining the information relating to the 
oksjectdata. 

According to a tiiirteentii aspect of the present 
invention, in the above-described demultiplexing appa- 
ratus, tiie information analysis means receives tiie 
object composition Information wttich is divided into at 
least one section, decides whether data of each section 
has a data error or not according to an error detection 
oode added to the end of each section, and discards tiie 
section having a data en^or. 

According to a fourteenth aspect of the present 
Invention, in tiie above-described demultiplexing appa- 
ratus, tiie information analysis means receives tiie infor- 
mation relating to the object data, which is divided into 
at least one section, decides whether data of each sec- 
tion has a data error or not according to an error detec- 
tion code added to the end of each section, and 
discards the section having a data error. 

According to a fifteenth a^ect of tiie present inven- 
tion, there is provided a demultiplexing method compris- 
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ing the st^s of: receiving a muKpIexed packet stream 
in which a plurality of object data are multiplexed, and 
separating packets containing N pieces of information 
relating to a program from the multiplexed packet 
stream with reference to packet identification numbers 
added to the respective packets; analy2dng the N pieces 
of irrfbrmation relating to the program, and obtaining a 
packet identification nunrt>er PID_syn of a packet con- 
taining o bject composition informat ion, and a packet 
identification n umber PIP o d of a' packet containing 
infonrialion relating to the o bject data : and separating 
packets, the packet identification nunijers of which 
matrfi the padcet id entification ijumbers RID sv n and 
P ID_od , from the multiplexed packet stream. 

According to a sixteenth aspect of the present 
inventbn, there Is provided a data recording medium in 
which a program Is recorded, and the program com- 
prises the steps of: receiving a multiplexed packet 
stream in which a plurality of object data are multi- 
plexed, and separating packets containing N pieces of 
information relating to a program from the multiplexed 
packet stream with reference to packet identification 
numbers added to the respective packets; analyzing the 
N pieces of information relating to the program, and 
obtaining a packet id entiflcatior ^, numt^er, pip ^y^^ rrf a 
packet oontairvng object composition information, and a 
packet Identification number PIP od of a packet con- 
taining information relating to the object data; and sepa- 
rating packets, the padcet identification numbers of 
which match the packet Identification numbers PIP.syn 
and PIP_od. from the multiplexed packet stream. 

BRIEF PESCRIPTIQN OF THE PRAWINGS 

Figure 1 is a block diagram illustrating a demulti- 
plexing apparatus according to a first en^xxliment 
of tiie invention. 

Figure 2 is a diagram illustrating a data format of a 
multiple}^ packet stream input to a demultiplexer 
according to the first embodiment. 
Figures 3(a)-3(d) illustrate data tables of packets in 
the multiplexed packet stream shown in figure 2 
and, more specifically, 3(a) shows a program asso- 
ciation table, 3(b) shows a program nnap tattle. 3(c) 
shows an Initial ot>iect desaiptor section, and 3(d) 
shows an object descriptor section. 
Figure 4 Is a block diagram illustrating a demulti- 
plexing apparatus according to a second embodi- 
ment of the Invention. 

Figures 5(a)^c) are diagrams for explaining a data 
recording rfiedium containing a program for Imple- 
menting a demultiplexing process according to the 
first or second embodiment, using a computer sys- 
tem. 
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Rgure 1 is a block diagram for explaining an appa- 
ratus f or demultiplexing multiplexed da ta (hereinafter 
referred to simply as "a d emuitiplexinq apparatu s") 
according to a first embodiment of the present inven- 
10 tion. 

As shown in figure 1 , the demultiplexing apparatus 
according to tiie first e mbodiment comprises a dem ulti- 
plexer 301 which receives a multiplexed.packet strea m 
In which a p lurality of object da ta are multiplexed, and 

75 demultplexes tiie multiplexed data with r eference_t o 
^ packet i dentification numbe rs given to the packets; a 
fi rst decoder 30 2 which de codes first obiect data sepa- 
rated from the multiplexed data and outputs first 
decoded object data; a second decode r 303 which • 

20 decodes second object dat a separated from tiie multi- 
-plexed data and outputs second decoded object data; a 
f irst memory 304 which stores a program association 
table (first infamation relating to a program) separated 
by the demultiplexer 301 ; a second menrory 305 which 

25 stores a program map table (second information relating 
to tiie program) separated by tiie demultiplexer 301 ; a 
t hird memory 3 06 whi(^ stores an Initial object descrip- 
tor section (tiiird information relating to the program) 
separated by the demultiplexer 301; a fourth memory 

30 307 which stores an o biect composition information 
section separated by tiie demuHiple)®r 301 ; a fifth mem - 
ory 308 which stor es an obiect d^criptor sectio n (infor- 
mation relating to tiie object data) separated by tiie 
demultqslexer 301 ; a conroosttOLaO^ which composites 

35 the first decoded obiect data and the second decod ed' 
o biectda ta: and an information analyzer 310 which ana- 
lyzes tiie pa cket identification numb ers from tiie respec- 
tive data stored in the first to fifth memories 304, 305. 
306, 307, and 308. 

40 Rgure 2 is a diagram showing a data format of tii e 
multiplexed packe t stream input to tiie demultiplexer 
301 . Refening to figure 2, tiie nrtultiple)@d nackftt stream 
is divided Into a pl urality of fixed^eno th packete which 
are defined as TS (Transport Stream) packets in 

45 M PEG2 . and e ach packet is given a packet identification 
n umber called PIPJPacket Identifie r). In a pad<et tiie 
PIP of which is J. a program association table (PAT) is 
stored. In a packet tiie PIP of which is IQp, a program 
map table (PMT) Is stored. In a packet the PIP of which 

so Is 1 10, an i nitial object descript or section (Initial OP sec- 
tion) is stored. In a packet the PJP^ of which is 101 , 
o bject data of MPEGg video is stored. In a packet tiie 
PIP of which is 102. obi ect data oTmpEGI audio is 
stored. In a packet the PIP of which is 103i an objec t 

55 composition information section Is stored. In a packet 
tiie PIP of which is 104, an o bject descrip tor section 
(OP section) is stored. In a packet tiie PIP of whidi is 
105, object data of MPEQ4 video (#1) Is stored. In a 
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packet t he PIP of which is 106. object data of MPE G4 
vi deo (#2y,is_stQced . In this first embodiment, the multi- 
plexed packet stream is demultiplexed with reference to 
the PIDs given to the re spective pad<ets . by analyzing 
the above-mentioned data. 5 

Figures 3(a)-3(d) are diagrams showing portions of 
the program association table (PAT) , the program twp 
table (PMT) . the Initial object descriptor section (Initial 
QBl: and the object descriptor section (00 section), 
respectively. The program association table (PAT) io 
shown in figure 3(a) has program numbers of the pro- 
gram map table, and identification numbers PMT_PID 
of packets corresponding to the respective program 
numbers. The p rogram map table (PMT) shown In figur e 
3(b) has streamTvpes as data ty pes, and identification is 
n umt)ers ES_PID of packets oonresDonding _tO-Jb e 
re spective stteani tvoes. The da ta type is information 
indicating the ki nd of data content of the packet sp eci- 
fi ed by the PIP . In figure 3(b), ES_PID_101 indicates 
tiiat the d ata type is MPEG2 Video ES_PID_102 Indi- zo 
cates that the data type is M PEG1 Audig and 
ES_PIO_110 indicates that the d ata type Is MPEG4 . 
The initial object descriptor section (Initial OP section) 
shown in figure 3(c) has stream types as data types, 
and identification numbers ES_PID of packets oorre- 25 
spending to the respective stream types. In figure 3(c). 
ES_PIP_103 Indicates that the data type is object com- 
position information (SP), and ES_PIP_104 indicates 
that the data type is object descriptor section (OP sec- 
tion). The object descriptor section (OP section) shown 30 
in figure 3(cl) has stream types as data types, and Iden- 
tification numt^rs ES.PID of packets corresponding to 
the respective stream typ^. The object descriptor sec- 
tion further Includes a stream map table (SMT) describ- 
ing ES_PIP Off MPEQ4 data although It is not shown In 35 
figure 3(d). In figure 3(d). ES_PIO_105 lndk»tes tiiat 
the data type is MPEQ4 Video (#1). and ES„PID_106 
indicates that the data type Is MPEQ4 Vtdeo (#2). 

Hereinafter, the operation of the demutttplexing 
apparatus accordir^ to the first embodiment will be ^ 
desaibed i^ng figures 1. 2. and 3(a)-3(d). 

When the demultiplexer 301 re ceives the mu lti- 
pl ied packet stream shown In figure 2, it separates ti e 
pactet the PIP of which is 0, from the stream. This 
packet is output as a program association table and 4S 
stored in the first memory 304. 

The information analyzer 310 receives the program 
association table from the first memory 304 and ana- 
lyzes this table. To be specific, the analyzer 310 
searches the table for a program number corresponding so 
to a program to be received by the demultiplexing appa- 
ratus, and obtains PMT__PIP_m which is a PIP value of 
a program map table associated to the program 
numt>er. Then, the information analyzer 310 outputs the 
PMT_PIP_m to the demultiplexer 301, In tfiis first ss 
embodiment. It is assumed that the analyzer 310 
obtains PMT_PID_100 shown In figure 3(a) as the 
PMT PIP m. 



Receiving tiie PMT_PIP__m, the demultiplexer 301 
s eparates a packet, the PIP of which notches . tiie 
MPT PIP nn„ fppp^ multiplexed packet strea m 
s hown in fiaure 2 fin this case, a packet-Jhavjn o 
PIP_1 00). This padtet is output as a program map table 
and stored in the second memory 305. 

The information analyzer 310 receives the program 
map table from the second memory 305. and analyzes 
tills table to obtain the PIP of packet of data related to 
tiie program to be received by the demultiplexing appa- 
ratus (in this case, ES_P1P_101 and ES_PIP_102 
shown in figure 3(b)). and tiie identification number 
PIP_iod of packet containing information for expressing 
the meaning in MPEQ4 (ES_PIP_110 shown In figure 
3(b)). Then, the irrfbrmation analyzer 310 outputs tiie 
ES_PIP_1 10 as the PIPJod value of ttie packet relating 
toMPEQ4. 

The demultiplexer 301 separates a packet, the PIP 
o f which matches the ES_PIP , 110^ from the^ multi- 
pl exed pad<et stream shownJn f igure 2. This packet is 
output as an initial object descriptor section and stored 
in the third memory 306. 

The information analyzer 310 receives the initial 
object descriptor section from the third memory 306. 
and analyzes this table to obtain tiie klentiflcation 
numk)er PIP_8d indicating that the data type Is the 
object composition infbnmation section (object composi- 
tion infbnrmtion data or scene description data) (in this 
case, ES_PIP_103 shown In figure 3(c)), and the identi- 
fication numt)er PIP_od indicating tiiat the data type is 
tiie object descriptor section (object descriptors) 
(ES_PID_104 shown in figure 3(c)). These PIPs are 
output to the demultiplexer 301 . 

The demultiplexer 310 separates the packet of the 
o bject composition Infbmrgbon section indicated b v tiie 
PIP_ sd and the packet of ttie.ob iecjLdescdPtor_section 
in dicated by ,the, PIP o d from .tiie_multi plexed packet 
stream shown in figure 2. and stores tiie object compo- 
sition information section and the object descriptor sec- 
tion in the fburtii memory 307 and the fifth memory 308. 
respectively. 

Next, the information analyzer 310 receives tiie 
object composition information from the object composi- 
tion information section stored in the fourth menrx)ry 
307, analyzes ttiis information, and outputs composition 
information, such as composition position and composi- 
tion order, toward the compositor 309. Further, tiie infor- 
mation analyzer 310 receives the object descriptor 
section from the f rftii menrtory 308, and analyzes tills 
section to obtain the data type Indicating video or audio 
data coded by MPEG4 from the SMT included in tills 
section. The analyzer 310 outputs tills data type to th e 
demultiplexer 30 1. In thfe first emtKxiiment. there are 
t wo pieces of data having this data type^ and thj^e data 
a re respectlvelv described as f i r; stobiectjdata and sec- 
ond object data. F or example, the PIP value of the first 
o^'ect^ata is described PIPjdl while the PIP value of 
tiie second object data Is described as PIP_d2. Refer- 
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ring to f^ure 3(d). it is assumed that the PIDjdl Is 
ES_PIDJ05, and the PID_d2 is ES_PIDJ06. 

The demultiplexer 30 1 separates the packets ind i- 
c ated"W P ^9-<j||, ^^^l^ ^^^j lrom the multiple xed 
packet stream stown inJi quce,.2L^d-autputs these s 
packets to the fi rst decoder 302 and t he secon d 
decoder 303 . respectively. To be spedfic. the first cbjed 
w/ data indicated by ES_PID_105 (PIDjdl) is output to the 
firg^g^Ser 302 to be decoded, and the second object 
da ta indica ted by ES_PIDJ06 (PID_d2) is output to the io 
sessnd decoder 303 to be decoded. 

In the compositor 309, the first and secon d 
de coded obiect data respectivelv decoded fa LtbeJirst 
and second decoders 302 and 303 are corm)os ited 
according to the composition infornati on. such as com- is 
position position and composition order, supplied from 
the in1brmalk)n analyzer 310. 

The demultiplexing apparatus according to the first 
embodiment of the invention has various effects as fol- 
lows. 20 

Since the information analyzer 310 obtains the 
PIDs in the initial OD section in adcfition to the PIDs of 
the respective data descrbed In the program map table, 
and outputs th^e PIDs toward the demultiplexer 301. 
reliat)le separation of MPEQ4 data is realized. 2s 

Further, since the P\D§ of the packe ts for transm it-^ 
ting the object composition Information and the obiec t 
de scriptors can be identified by the data types indicating 
th e meaning in MPEG4 desaibed in the initial obie ct 
/ d escriptor section, these PIDs can be identified w ith so 

MoreovB-. since the IDs of the object descriptors 
desaibed in the object oonrposition Information are 
compared with the IDs of the respective object descrip- 
tors in the object descriptor section, the object conposi- 35 
tion information can be associated with the object data 
designated by the object descriptor, whereby the object 
data separated by the demultiplexer 301 can be com- 
posited according to the composition position and the 
composition order described in the composrtbn infor- 40 
mation. 

[ Embodiment 21 

Figure 4 is a block diagram for explaining a demuiti- 4s 
plexing apparatus according to a second embodiment 
of the present invention. 

As shown in figure 4. the demultiplexing apparatus 
according to the second embodiment comprises a 
demultiplexer 1 which receives a multiplexed packe t so 
st ream in which a plurality of ob j ect data are mu lti- 
ple xed, and demultiplexes the multiplexed data with re f- 
ere nce to padiet identification numbers given to the 
pa ckets; a first decoder 2 which decodes first obiect 
da ta separated fr om tiie multiplexed data and out puts ss 
first d ecod^ object da ta ; a second dec oder 3 whi ch 
decoSes^second ^iect'c&ta^se c ^ fronTttTemu lti- 
plexed data and o utputs se cond de cod ed obiect data: a 



first memory 4 which stores a program association table 
(first Information relating to a program) separated by the 
demultiplexer 1 ; a second memory 5 which stores a pro- 
gram map table (second Information relating to the pro- 
gram) separated by the demultiplexer 1 ; a third memory 
6 which stores object composition information of an 
object composition information section separated by the 
demultiplexer 1 : a fourth memory 7 which stores object 
descriptors of an object descriptor section having infor- 
mation relating to the object data separated by tiie 
demutttplexer 1 ; a compositor 8 which composites ti ie 
fir st decoded object data and the second decoded ^^/^^ 
objert da ta; and an information analyzer 9 which ana- 
lyzes the packet identification nunrtbers from the first 
infbnnation relating to the program, the second informa- 
tion relating to ^e program, the object composition 
information, or the object descriptors. 

A descripticm is now given of the operation of tiie 
demultiplexing apparatus according to the seccmd 
embodiment 

TTie data input to tiie d emultiplexer 1 is a mutti- 
pl exed packet stream which is defined_as_a transport 
st ream (TS) in MPEG 2. Eac*i packet in tiie multiplexed 
packet stream is given a packet identification numt>er 
called PID (packet identifier). Initially, the demultiplexer 
1 separates a packet tiie PID of which is 0, and outputs 
tills packet as a program association table to be stored 
in the first memory 4. 

The Information analyzer 9 receives the program 
association tat)le from the first memory 4. searches the 
table for a program number corresponding to a program 
to received by tiie demultiplexing apparatus, and 
obtains PlD_m which is a value of a program map PID 
associated to the program number. Then, the analyzer 9 
outputs the PID_m to the demultiplexer 1 . 

Receiving the PID_m, the demultiplexer 1 sepa- 
rates a packet, the PID of which matches the program 
map PID_m. from the multiplexed packet stream, and 
outputs this packet as a program table to be stored in 
the second memory 5. 

The information analyzer 9 receives tiie program 
map table from the second memory 5, and obtains tiie 
PID of packet of data related to the program to be 
received by the demultiplexing apparatus. 

Further, the information analyzer 9 analyzes infor- 
mation for expressing the meaning in MPEG4. which 
information is described in relation to tiie PID of each 
data in tiie program map table. To be specific, tt\\s infor- 
mation comprises the data types in MPEG4. the object 
IDs of the MPEG4 objects described by the data types, 
and tiie PIDs of the MPEG4 padtets. T Tie data type Indi - 
ca tes the kind of data content of the packet specified bv 
ti ie PID . T here are ttiree kinds.of data CQntec|ts_as , tb l- 
lows : an object composition information section, an 
object descriptor section, and data otiier tiian tiiese 
sections. TTie information analyzer 9 analyzes tiie data 
type, and outputs the identification number PID_syn of 
tiie packet including tiie object composition information 
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(object composition information section) and the identi- 
fication number PID_od of the packet including the 
object descriptors (object descriptor section), toward 
the demultiplexer 1. 

When the data type detected by the information 
analyzer 9 from the program map table Is neither the 
object composition information section nor the object 
desalptor section, the information analyzer 9 obtains 
the object ID. When an MPEG4 FlexMux channel (FMC) 
is described, the Information analyzer 9 obtains this as 
well. The FMC means a packet identification numt>er in 
MPEG4 padcetization. In tills second embodiment, 
there are two pieces of data having tiie data type otiier 
than the above-mentioned sections, and these data are 
respectively descrbed as the first object data and tiie 
seoorKj object data. In the first object data, no FMC 
used. Its PID value is PID_d1 . and its object ID value is 
ODJdjdI. In the second object data, FMC_d2 is 
desaibed as an FMC value. Its RID value ts PID_d2, 
and its object ID value is 0DJdjd2. The information 
analyzer 9 outputs PIDjdt. PID_d2, and FMC_d2 
toward the denrtultiplexer 1 . 

Receiving PID_syn, PIDjod, PID_d1. PID_d2, and 
FMCjd2, the demultiple)cer 1 separates a packet the 
P ID of wrhich is RID syr^ (q^ifiPrt ppmpQsi^iQn informatinn 
se ction) from tiie multiPlexe dLoad^etJStceam^jarKLout- 
puts this pa(tet as object composition information to the 
third menrrory 6. F urtiier, the demultiplexer 1 se parates 
a packet the PID of which is PID o d (object descriptor 
section) fro m tiie multiplexed packetj stream, and out- 
puts this packet as object descriptors to the fourth mem- 
ory 7. Fur ther, the demultiplexer 1 seioarat es a packet, 
the PID of which is PID_d1, and outputs this packe t to 
th e first decoder 2 . Furthermore, the demultiplexer 1 
separates a packet, the PID of which is PID_d2 and th e 
F MC of which is FMC d2. and outPutsi bsjacketFoihe 
second decoder 3. 

Receiving the object composition information, the 
information analyzer 9 obtains information sudi as the 
composition position of olsjects and the hierarchy of 
objects, and the object IDs of the respective objects, 
and outputs these to the compositor 8. Furtiier. the 
informatk}n analyzer 9 receives an object descriptor to 
which tiie object ID value of the first object data, 
0DJd_d1. is given, and obtains a coding scheme 
Codejdl, a code AL_predefined_dl incficating whether 
the MPEQ4 packetization using the Access Unit Layer 
exists or not, and a cede AL_cortfig_d1 indicating set- 
ting information of tiie packetization. The information 
analyzer 9 analyzes these codes to create access-unit- 
layer instruction information AL_d1, and outputs ALjdl 
and Code.dl to the first decoder 2. Moreover, the infor- 
mation analyzer 9 receives an object descriptor to which 
the object ID value of tiie second object data, 
0DJd_d2. is given, and obtains a coding scheme 
Codejd2, a code ALj)redefined_d2 incficating whether 
the MPEQ4 packetization using the Access Unit Layer 
exists or not. and a code AL_config_d2 indicating set- 



ting information of tiie packetization. The information 
analyzer 9 analyzes these codes to create accessHinit- 
layer instniction information AL_d2, and outputs ALjd2 
and Code_d2 to the second decoder 3. 

5 Th e first decoder 2 decodes the first object data 

according to the coding scheme Codejdi. and outputs y 
tiie decoded data. The SQ fprpji^codiQ r 3 decodes tiie ^ 
second object data according to tiie coding scheme 
Code_d2, and outputs the decoded data. 

10 The comt>osito r 8 receives t he decoded first obje ct 
dat a and tiie decoded seoond o b ject data, and compos- 
ites ttie first and second o bject data witti reference to -/jr 
tiie o bject IDs and the Info rmati on relatinq .to^tiie.object 
hierarchy and the like, which are sif^lied from ttie infor- 

15 m ation ar alyzer 9, and outputs the composite.data. 

The above-mentioned demultiplexing apparatus 
according to ttie second emtxsdiment provides tiie fbl- 
kwing effects. 

Since the infbmnation analyzer 9 obtains the PID of 

so tiie MPEQ4 multiplexed packet in addtion to the PID of 
each data descrbed in the program map table, and out- 
puts this PID to tiie demultiplexer 1, reliable separation 
of the MPEG4 data Is realized. 

Further, since the PIDs of ^e packets for transmit- 

2ff ting the object composition information section and tiie 
object descriptor section are identified by tiie data types 
described in tiie program map table and Incficating the 
meaning in MPEG4. these PIDs can be identified witii 
reliability. 

30 Furthermore, since the object IDs in MPEG4 are 
obtained with respect to the PIDs of tiie respective data 
described in the program map tak3le. and these object 
IDs are compared with the object IDs desaibed in tiie 
object desaiptor section, the object IDs can be assod- 

35 ated with the PIDs. whereby tiie re^>ective data sepa- 
rated witii PIDs can be composited using the object IDs 
described in the object composition Information section. 

Moreover, when the Information analyzer 9 receives 
tiie object descriptors, it analyzes the code Indicating 

40 whetiier tiie MPE(34 Access Unit L^er is used or not 
and tiie code indicating the setting information, and out- 
puts ttie result of tiie analysis togetiier witii tiie Informa- 
tion indicating the coding scheme toward the decoder. 
So, ttie decoder can perform appropriate header 

45 processing of the Access Un'rt Layer and appropriate 
decoding, and this eneibles packetization using no 
MPEG4 Access Unit Layer. Accordingly, a broadcasting 
in which MPEG4 object data is rr\\xe6 can be received 
by the conventional MPE(3i2 receiver, while PES-pack- 

50 etized data can be received by the MPEGi4 receiver. 

In the first (second) embodiment described atx)ve, 
when ttie information analyzer 310 (9) receives the 
object composition information and tiie object desaip- 
tors to analyze the contents tiiereof, it may receive 

55 tiiese data to which version numbers, tiie values of 
which vary when ttie contents of ttiese data vary, are 
added as header information. In this casa the informa- 
tion analyzer 310 (9) stores the version ^numbers of 
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these data once analyzed and, when these data are 
input nest the version numbers of the input data are 
compared with the version numbers stored in the ana- 
lyzer 310 (9). Only when the version numbers do not 
matdi. the analyzer 310 (9) analyzes the data again. $ 
Thereby, the burden on data analysis by the information 
analyzer 310 (9) is significantly reduced. 

Further, the Information analyzer 310 (9) may 
receive the data obtained by dividing the object compo- 
sition information and the object descriptors into a plu- io 
rality of sections and giving a section number at the 
head of each section. In tiiis case, the Information ana- 
lyzer 310 (9) can restore the object oonposrtion Infor- 
mation and the abject descriptors by integrating the 
sections. is 

Furtiiermore. the information analyzer 310 (9) may 
receive the data to which an error detection code Is 
added at the end of each section. In this case, the infor- 
mation analyzer 310 (9) can check whether the data 
within each section includes a data enor or not, by refer- 20 
ring to the en'or detection code. As an error detection 
code, a CRC (Cyclic Redurxlancy Check) code or a 
checksum Is available. 

While in the first (secorxQ enrlkxxjlment of ttie inven- 
tion the demultiplexer 301 (1), tiie first and second 2s 
decoders 302 and 303 (2 and 3), the compositor 309 
(8). and the information analyzer 310 (9) are described 
as individual urvts. the present Invention is not restricted 
thereta In an actual hardware structure, ttiese units can 
be Implemented by at least one LSI or microprocessor. 30 
Furtiier, It is needless to say that the first to f iftii memo- 
ries can be implemented by at least one memory, such 
as a DRAM (Dynamic Random Access Memory) or a 
SDRAM (Synchronous Dynamic Random Access Mem- 
ory. 3S 

While in the first and second embodiments the^ 
demultiplexBrs 30 1 and 1 r eceive a multiplexed pack et 
str eam in which packets are multiplexed by MPEG 2, 
these demultiplexerB may receive m ultiplexed packe t 
str eamTSBfimSTbyotHy^^^ as a_!]:gj!^- ^ 

/ ^/o- Vy. plexing scheme defined in ITll-TH.223, or TCP (Trans- 
mission Control Protocol). UDP (User Datagram 
Protocol), and RTP (Realtime Transport Protoc ol), 
wh ich protocols are used in internet 

Moreover, while the d emultiplexing apparatus es 4s 
according to the fi rst and second embodiments ar e 
im plemented by hardwa re, ttie present invoDti on is not 
re stricted the reto. Th e demul tiplexer, th e first and sec - 
ond decoders, tiie compositor, and the informati on ana- 
lyzer according to the afor ementioned embodime nts so 
^r^^^ may' be'ffT^piimernecl asarsoftware.proQram in which 
the func tions of these units are, pro c framm ed. with tiie* 
same effects as described for the aforementioned 
embodiments. 

55 

[Embodiment 3] 

Figures 5(a)-5(c) are diagrams for explaining the 



case where the demultiplexing process according to tiie 
first or second embodiment Is inplemented by a compu- 
ter system using a floppy disk containing a program of 
thedamultiptexing process. More specifically, figure 5(b) 
shows a front view of a case F containing tiie floppy 
disk, a aoss^ectional view thereof, and the floppy disk 
FD (recording medium body). Figure 5(a) shows an 
example of a physical format of the floppy d^k FD. At 
the surface of the disk, a plurality of tracks Tr are formed 
oonc^ically from the outer circumfererKe of the disk 
toward tiie inner circumference. Each track Is divided 
into 16 sectors in the angular direction. Therefore, in tiie 
floppy disk containing tiie above-mentioned program, 
data of tiie program are recorded in assigned sectors on 
ttie floppy disk FD. 

Rgure 5(c) shews the structure for reootding/repro- 
ducing tiie program In/from tiie floppy disk FD. When 
tiie program is recorded in the ftoppy disk FD, data of 
the program are written in the floppy disk FD from the 
computer system Cs through tiie f toppy dsk drive FDD. 
When tiie above-mentioned demultiplexing apparatus is 
constructed in the computer system Cs from the pro- 
gram recorded in the floppy disk FD, the program is read 
from the floppy disk FD by the floppy disk drive FDD and 
tiien loaded to tiie computer system Cs. 

As descra>ed above, when a program for imple- 
menting tiie present invention by software Is recorded in 
a data recording medium such as a floppy disk, demul- 
tiplexing of multiplexed data according to tiie aforemen- 
tioned ent)Odiments can be easily e)®cuted in an 
Independent general-purpose computer system. 

While in this third emtxxliment a floppy disk is 
employed as a data recording medium, an optical disk 
may be employed in simitar way Further, applicat)le 
recording media are not restricted thereto, namely, any 
recording media, such as an IC card and a ROM cas- 
sette, may t>e employed as long as the above-men- 
tioned program can be recorded tiierein. 

Claims 

1 . A demultiplexing apparatus comprising : 

demultiplexing means for receiving a multi- 
plexed padcet stream In which a plurality of 
object data are multiplexed, and denxiltiptexing 
the multiplexed data with reference to packet 
identification numbers added to the respective 
packets; 

decoding means for decoding M pieces of 
object data separated from the multiplexed 
data, and outputting M pieces of decoded 
object data; 

a memory for storing N pieces of information 
relating to a program and separated by the 
demultiplexing means; 

a memory for storing object composition infor- 
mation separated by tiie demultiplexing means; 
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a memory for storing information relating to the 
object data and separated by the demultiplex- 
ing means; 

composition means for compositing the M 
pieces of decoded object data; and s 
information analysis means for analyzing the 
packet identification numbers from the N 
pieces of information relating to the program, 
the object composition informatioa or the infor- 
mation relating to the object data. 10 

A demultiplexing apparatiG as defined in Claim 1 
wherein: 

said information analysis means analyzes the is 
N pieces of Infornrmtion relating to the program, 
obtains a packet identification number PID_syn 
of a packet containing the ok^ect composition 
information, and outputs the packet identifica- 
tion number PID_syn to the demultiplexing 20 
means; and 

said demultiplexing means separates the 
packet containing the object composition infor- 
mation from the input multiplexed packet 
stream with reference to the packet dentif ica- 26 
tion number PID_syn. and outputs this packet. 

A demultiplexing apparatus as defined in Claim 1 
wherein: 

30 

said Information analysis means analyzes tiie 
N pieces of infbrnrration relating to the program, 
obtains a packet identificatbn numt>er PID_od 
of a packet containing the information relating 
to the object data, and outputs the packet iden- 35 
tification number PtD_od to the demultiplexing 
means; and 

sad demultiplexing means separates tiie 
packet corrtaining the informatfon relating to 
the object data from the input multiplexed 40 
packet stream with reference to the packet 
dentification number PID.od, and outputs this 
packet. 



A demultiplexing apparatus as defined in Claim 1 
wherein: 
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said information analysis means analyzes tiie 
Information relating to the object data, judges a 
code Al j)redeflned tor each of the object data, 
whic^ code indicates whether packetization 
according to the international standard MPEQ4 
exists or not, and outputs tiie code 
Al_predefined towand the demultiplexing 
means; and 

said demultiplexing means performs separa- 
tion of the MPEQ4 packetization for each 
object data witii reference to tiie code 
Al jDredefined. 

A demultiplexing apparatus as defined in Gaim 1 
wherein: 

said information analysis means analyzes the 
information relating to the object data, judges a 
code A! JDredefined tor each of the object data, 
whic^ code indicates whether peicketization 
according to the international standard MPEG4 
exists or not, and outputs tiie code 
Alj3redefined towcird the decoding means; 
and 

said decoding means pertornr^ separation of 
the MPEGi4 packetization with reference to the 
code Al^predefined. 

A demultiplexirrg apparatus as defined in Claim 1 
wherein: 

said information analysis means analyzes the 
information relating to the object data, judges a 
code AI_config for eac^ of the object data, 
which code indicates setting information of a 
packet header according to the international 
standard MPEG4. and outputs the code 
Aljoonftg toward the demultiplexing means; 
and 

said demultiplexing means performs separa- 
tion of tiie MPEG4 packetizatfon for each 
object data witii reference to tiie code 
AI_oonfig. 

A demultiplexing apparatus as defined in Claim 1 
wherein: 



said Information analysis means analyzes tiie 
object composition information, and outputs 
information indicating the hierarchical relation- so 
ship of the object data toward tiie composition 
means; and 

said composition means composites the M 
pieces of decoded object data with reference to 
the hi erarchical relationship. ss 

A demultiplexing apparatus as defined in Claim 1 
wherein: 



said information analysis means analyzes ttie 
information relating to the object data, judges a 
code Al_config for each of tiie object data, 
which code indicates setting information of a 
packet header according to tiie international 
standard MPEG4. and outputs the code 
AI_config toward the decoding means; and 
said decoding means perfornrts s^aration of 
the MPEG4 packetization with reference to tiie 
code AI_config. 
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9. A demultiplexing apparatus as defined in Claim 1 
wherein: 

said Information analysis means analyzes the 
N pieces of information relating to the program, 
ot}tains a packet identification numt>er PIDjd of 
a packet containing the object data and a 
packet identification number FMCjd according 
to the international starxiard MPEQ4, and out- 
puts these identification numbers toward the 
demultiplexing means; and 
said demultiplexing means separates the 
packet containing the object data from the input 
multiplexed padcst stream with reference to the 
identification numbers PID_d and FMCjd, and 
outputs the packet. 

10. A dennultiplexing apparatus as defined in Claim 1 
wherein said infomnation analysis means obtains a 
code for deciding whether the packet kientif ication 
numt>er FMCjd according to the international 
standard MPEG4 is added or not, by analyzing the 
N pieces of information relating to the program. 

11. A dennultiplexing apparatus as defined in Claim 1 
wherein said information analysis means receives 
the object composition Information which is divided 
into at least one section, and integrates these sec- 
tk)ns with reference to version numbers and section 
numk)ers which are added at the heads of the 
respective secttons. thereby obtaining the object 
corrqsosition Infbmiation. 

12. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means recoves 
the information relating to the object data, which 
information is divided into at least one sectton, and 
integrates these sections with reference to version 
numbers and section numbers which are added at 
the heads of the respective sections, thereby 
obtaining the information relating to the object data. 

13. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means receives 
the object composition information which is divided 
into at least one section, deckles whether data of 
each section has a data error or not acoording to an 
error detection code added to the end of each sec- 
tk)n, and discards the section having a data en^or. 

14. A demultiplexing apparatus as defined in Claim 1 
wherein said information analysis means receives 
the information relating to the object data, which is 
divided into at least one sectbn, decides whether 
data of each section has a data error or not accord- 
ing to an error detection code added to the end of 
each section, and discards the sectk>n having a 
data error. 
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15. A demultiplexing method comprising the steps of: 

receiving a nrtultiplexed padcet stream in which 
a plurality of object data are multiplexed, and 
separating packets containing N pieces of 
information relating to a program from the mul- 
tiplexed packet stream with reference to packet 
Identification numbers added to the respective 
packets; 

analyzing the N pieces of information relating 
to the program, and obtaining a packet identifi- 
cation number PID_syn of a packet containing 
object composition informatksn. and a packet 
identification number PIDjod of a packet con- 
taining information relating to the object data; 
and 

separating packets, the packet Identification 
numbers of which match the packet Identifica- 
tion numbers PID_syn and PIDjod, from the 
multiplexed packet stream. 

16. A data recording medium in which a program is 
recorded, said program comprising the steps of: 

receiving a multiplexed packet stream in which 
a plurality of object data are multiple)^, and 
separating packets containing N pieces of 
Information relating to a program from the nnil- 
tiplexed packet stream with reference to packet 
identification numbers added to the respective 
packets; 

analyzing the N pieces of information relating 
to the program, and obtaining a packet identifi- 
cation number PID_syn of a packet containing 
ot^ect composition information, and a packet 
identification numl^er PIDjod of a packet con- 
taining Information relating to the object data; 
and 

separating packets, the packet identificatk)n 
numfcmrs of which match the packet identifica- 
tion numbers PID.syn and PIDjod. from the 
nrujttiplexed packet stream. 
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Fi9*3 (d) program association table (PAT) 
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Fig. 3 (b) program map table (PMT) 
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Fig. 3 (c) initial object descriptor section (Initial OD section) 



stream type 


ES_PID 


SD Stream 
OD Stream 


103 
104 



FiQ.S (d) object descriptor section (OD section) 



stream type 


ES_PID 


MPEG-4 Video 
MPEG-4 Video 


105 
106 
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Fig .4 
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Fig.5 (a) 




Fig.5 (b) 
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